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INTRODUCTION 


CoBBamication  has  long  been  the  concern  of  man  since  it  is  the  basis 
of  his  learning  behavior.  Without  a  refined  communication  system  man  could 
not  educate  the  members  of  his  society  and  transfer  what  he  has  learned  to 
other  generations.    Language  is  the  focal  point  of  this  concern  but  the 
study  of  language  systems  in  isolation  produces  limited  advancement  in 
man's  knowledge  about  communication.    In  order  for  man  to  learn  about  his 
own  communication  he  must  learn  about  his  own  behavior.   More  specifically, 
he  must  learn  about  his  own  understanding  of  the  language  with  which  he 
communicates, 

A  major  problem  that  has  stood  in  the  way  of  this  understanding  of 
language  is  its  measurement.   The  problem  of  measuring  human  language 
behavior  has  been  a  difficult  one,  and  now  is  a  concern  of  several  dis- 
ciplines such  as  Speech,  English,  Psychology  and  Linguistics,   Much  pro- 
gress has  been  made  in  the  specific  area  of  reading  comprehension,  but  new 
and  better  methods  are  constantly  being  sought.   Of  the   many  questions 
that  have  been  asked  about  measurement  of  language  understanding  probably 
the  most  important  is:  how  may  a  test  be  devised  to  measure  how  well  the 
reader  understands  what  he  reads?   In  order  to  answer  this  questions 
another  must  also  be  asked:  does  the  type  of  material  read  influence  the 
reader's  understanding?   These  two  questions  have  been  answered  fairly 
well  by  many  scholars  in  the  past,1    In  1953  Wilson  Taylor  developed  a 


*Jm  Lorge,  "  Readability  Formulae  -  An  Evaluation,"  Elementary  English, 
XXVI,  (19h9),  PP.  86-95. 

Jtadolf  flesch,  "A  New  Readability  Yardstick,"  Journal  of  Applied 
Psychology,  XXXII,  (June  19ii8),  pp.  221-233. 
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different  type  of  measurement  which  answered  the  above  questions,  but  in  a 
simplified  test  called  cloze  procedure, 

STATEMENT  OF  THE  PiDHLEM 

The  first  objective  of  this  report  is  to  add  or  subtract  support  to 
Taylor's  findings  that  cloze  procedure  is  an  effective  measurement  of  read- 
ing comprehension.   This  will  be  done  by  determining  whether  or  not  cloze 
procedure  measures  the  reading  comprehension  of  a  person   reading  a  short 
expository  article,    further  support  or  non-support  will  be  given  to 
Taylor's  findings  by  determining  whether  or  not  cloze  procedure  and  a  multi- 
ple choice  test  over  the  same  article  correlate  significantly* 

The  second  objective  of  this  report  is  to  explore  the  application  of 
cloze  procedure  to  testbook  reading  assignments.   This  will  be  accomplish- 
ed by  determining  whether  or  not  cloze  procedure  measures  the  reading  com- 
prehension of  a  person  reading  several  chapters  of  a  textbook, 

DEFINITIONS 

In  order  to  accomplish  the  above  objectives  several  definitions  must 
be  set  forth  for  clarity.   First,  a  short  expository  article  is  defined  as 
an  article  where  a  ^00-word  cloze  passage  represents  approximately  fifteen 
per  cent  of  the  total  words  in  the  complete  article.    In  this  3>00-word 
passage  every  tenth  word  will  be  deleted,  thus  creating  fifty  blanks  in  the 
500-word  cloze  passage.   An  extended  textbook  assignment  is  defined  as  the 
number  of  chapters  in  a  textbook  where  a  £00-word  cloze  passage  represents 
approximately  four  per  cent  of  the  total  words  in  the  textbook  chapters. 
Every  tenth  word  will  be  deleted  in  the  textbook  chapters,  again  creating 
fifty  blanks  in  the  500-word  cloze  passage. 
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The  third  definition  is  that  of  cloze  procedure  itself.   Cloze  procedure 
derives  its  name  from  Qestalt  psychology,   "CLozure"  is  the  human  tendency  to 
close  the  gap  between  that  which  is  known  and  the  unknown.   An  example  of 
this  is  an  automobile  extended  halfway  from  a  garage,   A  person  looking  from 
the  side  can  see  only  half  of  the  automobile,  but  presumes  what  the  half  of 
the  car  in  the  garage  looks  like  from  his  previous  experience  with  auto- 
mobiles.   In  other  words,  the  person  looking  at  the  car  is  closing  the  gap 
between  the  known  and  the  unknown.   The  same  phenomenon  takes  place  with  the 
reader;  if  one  word  in  a  sentence  is  deleted,  a  word  is  chosen  by  the  read- 
er to  fill  in  for  the  deleted  word.   The  substitute  word  comes  from  the 
reader's  wide  experience  with  the  language,  and  more  specifically  from  the 
actual  content  of  the  paragraph  and  sentence  that  the  person  is  reading. 
Therefore,  the  reader  with  a  clear  understanding  of  the  whole  article  which 
he  is  reading  is  more  apt  to  duplicate  the  word  that  has  been  deleted. 

"The  better  the  writing  is  understood,  the  more  likely  it  is  that  a  reader 
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can  guess  what  words  are  missing," 

doze  procedure  was  originally  designed  to  test  the  readability  of  * 
particular  piece  of  writing,  but  it  was  soon  recognized  that  if  an  article 
was  "readable"  then  it  must  also  be  "understandable"  to  the  reader.    If  the 
reader  understands  what  he  reads  then  this  must  be  a  piece  of  writing  that 
is  "comprehensible"  to  the  reader.   Therefore,  if  "understanding"  and 
"comprehensionare  synonymous,  "readability"  is  synonymous  with  the  other 
two.   This  is  further  explained  by  Taylorx 


Wilson  L.  Taylor,  "Cloze  Headability  Scores  as  Indices  of  Indivi- 
dual Differences  in  Comprehension  and  Aptitude,"  Journal  of  Applied 
Psychology,  XLI,  fto.  1,  (1957),  p.  19.  


"It  Is  assumed  at  the  outset  that  readability  and 
comprehensibility  are  essentially  synonymous  terns* 
The  statement  that  a  particular  prose  passage  is, 
for  example,  'very  readable1  simply  seems  to  mean 
that  it  is  very  easy  for  some  'average1  reader  to 
understand  ."3 


The  operation  of  a  cloze  procedure  test  is  rather  simple,  but,  as  will 
be  discussed  later,  its  ability  to  measure  reading  comprehension  seems  com- 
parable to  the  other  methods  that  have  been  devised.   To  begin  a  cloze  pro- 
cedure test  an  article  or  reading  passage  is  chosen  and  assigned  for  the 
student  to  read.   After  the  student  has  read  the  article  a  passage  from  the 
article  is  chosen  at  random,  and  every  tenth  word  is  deleted.    In  Taylor's 
studies  he  chose  a  passage  that  was  approximately  twenty  per  cent  of  the 
complete  article.   The  student  is  then  asked  to  fill  in  the  blanks  with  the 
correct  words.   An  example  of  a  cloze  test  is  as  follows: 

"Mary  had  a  little  lamb, 

its  fleece  was  white     1     snow,  1,  _______ 

And  everywhere  that  Mary  went 

the  lamb  was     2     to  go,  2,  

It  followed  her  to  school  one  day, 


to  see  the  lamb  at  school  V 

Each  exact  reproduction  of  the  deleted  word  is  scored  as  one  correct 
answer;  synonyms  are  not  correct,    Mow  the  scores  can  be  compared  to  other 


3. 


It  made  the  children  laugh  and  h 


Taylor,  p,  19. 
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students  who  have  taken  the  test  to  determine  how  much  the  student  under- 
stands of  the  article  he  read.    In  the  selection  of  "Mary  Had  a  Little  Lamb" 
the  cloze  score  would  be  near  perfect,  but  from  a  physics  textbook  the 
scores  would  naturally  run  lower.   However,  the  student  who  understands  the 
passage  better  would  receive  a  higher  cloze  score  than  the  student  who  does 
not  understand  what  he  is  reading.   As  can  be  seen,  the  constructing  and 
administering  of  this  test  is  relatively  straight-forward. 

LIMITATIONS 

The  multiple  choice  test  that  was  correlated  with  the  cloze  procedure 
scores  posed  a  limitation  upon  this  study.    It  must  be  assumed  that  the 
multiple  choice  test  is  a  valid  measurement  of  comprehension.   This  test 
over  the  short  expository  article  was  given  to  a  group  of  thirty-eight  high 
school  seniors.   Each  question  was  ranked  according  to  the  number  of  times 
it  was  missed  by  the  high  school  students.   Of  the  original  forty-three 
questions,  twenty-five  were  selected  for  use.   Eighteen  questions  were 
drawn  from  the  center  of  the  distribution  of  ranks.   Three  questions  were 
selected  from  those  questions  missed  by  almost  all  students  and  four  ques- 
tions were  selected  from  those  questions  missed  by  nearly  all  students. 

H3VTEW  OF  LITEEATUHE 

A  review  of  the  literature  revealed  that  use  of  the  cloze  procedure 
falls  into  three  major  categories:  cloze  procedure  as  a  measure  of  read- 
ability, as  a  measure  of  reading  comprehension,  and  as  a  testing  device  to 
measure  comprehension  in  other  types  of  studies.   These  three  categories 
follow  a  chronological  order  in  the  progression  from  a  simple  testing  de- 
vice to  measure  readability  to  a  significant  index  of  reading  comprehension. 
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When  cloze  procedure  was  first  developed  by  Wilson  Taylor  it  was  devised 
as  *  measure  of  readability  to  improve  upon  the  accuracy  of  other  readability 
formulas  such  as  the  Flesch  Formula.   But  it  was  soon  discovered  that  close 
procedure  measured  a  much  more  important  factor  in  readability  -  the  human 
factor  of  comprehension. 

The  Flesch  readability  formula  as  an  example  of  the  readability  for- 
mulas measured  readability  as  follows!  it  compounded  (1)  average  sentence 
length  in  words,  (2)  average  word  length  in  syllables,  (3)  average  percen- 
tage of  "personal  words",  and  (h)  average  percentage  of  "personal  sentences" j 
the  score  from  this  formula  amounted  to  the  readability  score  of  the  parti- 
cular piece  of  writing.^   The  formula  was  first  applied  to  a  reading 
passage  and  then  checked  by  a  multiple  choice  test  on  the  level  that  the 
formula  predicts.   This  step  was  further  explained  by  Flesch  just  after  he 
revised  his  readability  formula. 


"The  scores  computed  by  this  formula  have  a  range 
of  0  to  100  for  almost  all  samples  taken  from 
ordinary  prose.   A  score  of  100  corresponds  to 
the  prediction  that  a  child  who  has  completed 
fourth  grade  will   be  able  to  answer  correctly 
three-quarters  of  the  test  questions  to  be  asked 
about  the  passage  that  is  being  rated}  in  other 
words,  a  score  of  100  indicates  reading  matter 
that  is  understandable  for  persons  who  have  com- 
pleted fourth  grade  and  are,  in  the  language  of  , 
the  U.  S.  Census,  'barely  functionally  literate*  ."■ 


As  can  be  seen  the  human  comprehension  factor  has  been  omitted  by  this 
type  of  readability  measure,  and  this  very  factor  was  what  Taylor  tried  to 


lBudolf  flesch,  "A  New  Readability  Yardstick,"  p.  221. 
'Flesch,  p.  223. 
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initiate  in  his  cloze  procedure  measure  of  readability.  This  factor  is  dis- 
cussed by  Taylor  in  his  original  article  on  cloze  procedure, 

"Cloze  procedure  counts  no  such  elements.    It  seems , 
however,  to  measure  whatever  effects  elements  have 
on  readability.   And  it  does  at  the  same  time  that 
it  is  also  taking  account  of  the  influences .of  many 
other  factors  readability  formulas  ignore.*" 

As  may   be  seen  above,  cloze  procedure  takes  into  account  all  of  the 
factors  that  affect  readability,  including  the  human  comprehension  factor. 
Thus,  cloze  procedure  is  a  much  better  measure  of  readability.  However, 
while  developing  cloze  procedure  as  a  measure  of  readability  Taylor  dis- 
covered that  cloze  procedure  could  also  be  used  for  many  other  types  of 
measurement.   The  main  one  of  concern  here  is  that  of  reading  comprehension. 
After  the  inclusion  of  the  human  factor  of  readability  by  cloze  procedure 
the  stage  was  set  for  an  effective  measure  of  reading  comprehension.  This 
discovery  was  commented  on  in  Taylor's  original  article  dealing  with  cloze 
procedure. 

"If  future  research  substantiates  the  results  so  far, 
this  tool  seems  likely  to  have  a  variety  of  applica- 
tions, both  theoretical  and  practical,  in  other  fields 
involving  communication •"? 

In  a  later  article,  Taylor  announced  cloze  procedure's  ability  to 
measure  reading  comprehension. 


^Wilson  L.  Taylor,  "CLoze  Procedures  A  New  Tool  For  Measuring 
tteadability,  w  Journalism  Quarterly,  (Fall  1953),  p.  Ul7. 

^Taylor,  p.  Ulf>. 


8 


"The  technique  appears  to  be  an  effective  guage 
of  'individual  difference'  in  the  comprehension 
of  readers ."° 

Since  readability  and  comprehension  are  synonymous,  Taylor  was  ready  to 
test  cloze  procedure  as  a  measure  of  reading  comprehension  because  his 
rationale  was  now  complete. 

The  rationale  that  cloze  procedure  measures  comprehension  is  a  simple 
one,  and  yet  quite  logical.   There  are  many  factors  that  influence  a  reader's 
comprehension*    Soma  of  them  can  be  measured  by  certain  methods,  but  others 
cannot,   An  example  of  one  of  these  factors  that  is  difficult  to  measure  is 
the  reader's  enthusiasm.    If  a  reader  is  enthusiastic  and  wants  to  read  and 
understand  the  passage  that  is  being  read,  then,  presumably,  he  will  under- 
stand more  of  what  he  reads  than  if  he  lacked  enthusiasm.    Cloze  procedure  is 
able  to  measure  enthusiasm  by  asking  the  reader  to  duplicate  exactly  what  he 
read.   Thus,  the  enthusiastic  reader  should  score  higher  on  a  cloze  proce- 
dure test  because  his  enthusiasm  allowed  him  to  comprehend  more  of  what  he 
read,   Taylor  further  explained  this  point  in  his  article  written  in  1953, 

"The  total  context  of  any  language  behavior 
includes  everything  that  tends  to  motivate, 
guide,  assist  or  hinder  that  behavior.  It 
includes  verbal  factors  -  grammatical  skills 
and  multitudes  of  symbols  -  and  non-verbal 
ones  such  as  fears,  desires,  past  experiences 
and  intelligence."? 

All  of  these  intangible  factors  can  be  measured  by  cloze  procedure 

Wilson  L.  Taylor,  "ftecent  Developments  in  the  Use  of  Cloze  Proce- 
dure," Journalism  Quarterly,  XXXIII,  (Winter  193>6),  p.  1*2. 

9Taylor,  "A  New  Tool,"  p.  1*18. 
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because  of  its  duplication  method  of  testing  comprehension. 

Therefore,  by  asking  the  reader  to  duplicate  what  he  has  read  the  read- 
er demonstrates  what  he  has  understood,  and  reading  comprehension  has  been 
measured* 

Taylor  has  tested  cloze  procedure  extensively,  and  all  of  the  tests  show 
that  reading  comprehension  is  measured  significantly.   The  first  two  experi- 
ments were  reported  in  19S3  and  the  results  seemed  to  uphold  his  rationale* 
At  this  point  in  the  testing  of  cloze  procedure  it  appeared  that,  as  a  mea- 
sure of  reading  comprehension,  cloze  procedure  was  a  success. 


"Potentially  important,  it  seems,  is  the  fact 
that  a  cloze  score  appears  to  be  a  measure  of 
the  aggregate  influences  of  'all  factors' 
which  interact  to  affect  the  degree  of  cor- 
respondence between  the  language  patterns  of 
transmitter  and  receiver ."10 


The  second  report  by  Taylor  was  in  19$6  and  showed  that  after  more 
extensive  experiments  cloze  procedure  was  substantiated  even  more  as  an 
effective  measure  of  reading  comprehension. 


"The  results  fulfilled  expectations.  Entered 
in  Table  I  are  the  correlation  coefficients 
among  five  sets  of  scores  for  Ij8  subjects. 
All  the  coefficients  are  large,  and  all  are 
'significant'  in  the  sense  that  they  could  not 
be  so  large  by  pure  chance  even  once  in  a 
thousand  times."1! 


The  final  set  of  experiments  conducted  by  Taylor  were  quite  extensive 


10Taylor,  "A  New  Tool,"  p.  Ii32* 
Baylor,  "Be cent  Developments,"  p.  US>» 


as  they  attempted  to  substantiate  even  farther  the  ability  of  cloze  procedure 
to  measure  reading  comprehension*   This  set  of  tests  was  correlated  with 
multiple  choice  tests  that  measure  comprehension,  and  with  the  Armed  Force* 
Qualification  Test,  which  is  a  general  aptitude  test  of  innate  ability  to 
comprehend*   The  close  procedure  correlated  significantly  with  the  other 
multiple  choice  tests  as  stated  by  Taylor 

"If  the  comprehension  tests,  adopted  as  criteria* 
really  did  index  (a)  knowledge  of  the  article's 
content  before  it  was  read*  (b)  the  relative 
amount  of  that  content  remembered  immediately 
after  the  study*  (c)  the  increase  in  knowledge 
of  the  content  brought  about  by  study,  and  (d) 
general  aptitude  or  ability  to  understand,  in 
agreement  with  another  criterion,  the  AFQT,  then 
it  appears  that  cloze  scores  did  too**-^ 

In  summary,  then,  the  experiments  with  cloze  procedure  upheld  all 
expectations  and  proved  Taylor's  rationale  as  defensible.    It  seems  that  the 
number  of  experiments  and  the  number  of  subjects  in  these  experiments  were 
adequate  to  substantiate  Taylor's  rationale. 

Cloze  procedure  has  been  used  in  other  studies  that  deal  with  compre- 
hension} this  shows  its  attractiveness  as  a  coisprehensive  measure.  The 
acceptance  by  other  scholars  as  they  used  it  in  their  studies  shows  the 
confidence  placed  in  cloze  procedure,  and  in  Taylor's  methods  of  experi- 
mentation and  substantiation. 

One  of  these  studies  conducted  by  Darnell  on  the  relationship  between 
sentence  order  and  comprehension  used  cloze  procedure  to  measure  the  compre- 
hension that  took  place  after  changing  the  sentence  order  within  a  message* 


•^Taylor,  "doze  Iteadability  Scores,"  p*  25* 
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In  the  concluding  paragraph  of  this  study  the  author  discussed  his  use  of 
cloze  procedure* 


"The  strength  which  the  present  experiment  illus- 
trates is  the  usefulness  of  the  deletion-completion 
technique  (cloze  procedure),  it  can  be  used  in 
further  investigations  of  the  effect  of  sentence 
order  upon  comprehension."^ 


King's  study  used  cloze  procedure  as  a  measure  of  comprehension;  this 
involved  a  study  of  ethos  which  showed  that  cloze  procedure  is  being  accept- 
ed and  used  as  a  measuring  device  for  reading  comprehension. 


"The  validity  and  reliability  of  the  cloze  pro- 
cedure technique  as  a  measure  of  recall  has 
been  empirically  demonstrated  in  research  by 
Taylor."*** 


Another  study  by  Welden  used  cloze  procedure  to  measure  retention, 
which  seems  to  show  that  cloze  procedure  has  been  accepted  as  a  testing 
device  for  measuring  variables  other  than  reading  comprehension. 


"Jtetention  of  the  experimental  messages  was 
indexed  by  the  'cloze  procedure'  .  •  •  •  • 
This  procedure  was  selected  primarily  be- 
cause it  permits  the  use  of  realistic  rates  - 
sages •"15 


Donald  K.  Darnell,  "The  Relation  Between  Sentence  Order  and  Compre- 
hension," Speech  Monographs,  XXX,  (June  1963),  P«  100. 

^^Thomas  R.  King,  "An  Experimental  Study  of  the  Effect  of  Ethos  Upon 
the  Immediate  and  Delayed  Recall  of  Information,"  Central  States  Speech 
Journal,  XVH,  (February  1966),  p.  26. 

^erry  A.  Welden,  "The  Effect  on  Attitudes  and  Retention  of  Message 
Order  in  Controversial  Material,"  TJnpubl.  Diss.,  (Michigan  State,  1961), 
PP.  3£-36. 
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In  summary  of  the  previous  literature,  it  was  found  that  cloze  procedure 
has  advanced  from  a  readability  measure  to  an  accepted  method  of  measuring 
reading  comprehension.    Cloze  procedure's  acceptance  has  been  shown  by  its 
success  in  measuring  comprehension  as  well  as  its  acceptance  by  other  scholars 
as  they  have  used  it  as  a  measuring  device  in  other  studies. 

HYPOTHESES 

The  purpose  of  this  report  is  to  test  the  following  hypotheses.  The 
first  two  are  defined  as  replication  hypotheses  to  add  support  to  Taylor's 
earlier  findings. 

I.  Cloze  procedure  scores  measure  reading  compre- 
hension of  a  given  short  expository  article. 

II.  Cloze  procedure  scores  correlate  significantly 
with  a  multiple  choice  test  over  a  given  short 
expository  article. 

The  next  hypothesis  is  defined  as  an  exploratory  hypothesis  in  order 
to  determine  the  applicability  of  cloze  procedure  to  textbook  assignments. 

III.  Cloze  procedure  scores  measure  reading  compre- 
hension over  a  given  extended  textbook  assign- 
ment. 

JUSTIFICATION 

The  justification  of  this  study  is  three-fold.    First,  more  weight 
should  be  added  to  Taylor's  findings  through  the  replication  of  one  of  his 
experiments.    Cloze  procedure  as  a  measure  of  reading  comprehension  is 
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generally  accepted,  but  investigation  of  the  correlation  of  cloze  procedure 
scores  with  multiple  choice  test  scores  is  limited  to  Taylor's  early  work, 
Taylor  consistently  found  correlations  of  above  .70  in  his  experiment.  EQr 
replicating  the  correlation  section  of  Taylor's  experiment  more  weight  should 
be  added  to  the  substitutability  of  cloze  procedure  for  other  objective  tests 
of  comprehension. 

Secondly,  the  exploratory  section  of  this  report,  that  of  testing  cloze 
procedure  for  an  extended  passage,  should  give  some  insight  into  whether  or 
not  cloze  procedure  can  be  used  for  textbook  assignments.   The  main  limita- 
tion of  cloze  procedure,  as  repeatedly  demonstrated  by  Taylor,  seemed  to  be 
that  at  the  fifteen  per  cent  level  of  cloze  testing  a  test  would  have  to  be 
given  over  every  chapter  in  a  textbook  assignment.   The  fifteen  per  cent 
level  of  cloze  testing  is  when  the  cloze  passage  represents  fifteen  per  cent 
of  the  entire  number  of  words  that  could  be  found  in  the  assignment  being 
tested.   Thus,  the  msxlman  number  of  words  that  could  be  tested  at  the 
fifteen  per  cent  level  is  about  3,1*00,  which  forces  the  cloze  test  to  be 
given  quite  often  as  far  as  textbook  assignments  are  concerned. 

In  order  to  determine  whether  or  not  the  fifteen  per  cent  level  is  an 
Inherent  limitation  of  the  cloze  procedure  the  small  level  of  four  per  cent 
was  explored  in  this  report.   The  four  per  cent  level  was  arbitrarily  chosen. 
It  resulted  from  the  length  of  the  first  assignment  in  the  course  from  which 
the  subjects  were  drawn.   The  four  per  cent  level  should  in  effect  show 
whether  or  not  cloze  procedure  is  acceptable  for  textbook  assignments. 

Finally,  the  practical  advantages  of  cloze  procedure  seem  to  justify 
this  report  as  these  practical  advantages  appear  to  weigh  heavily  in 
choosing  cloze  procedure  over  other  methods  of  measuring  reading  compre- 
hension.  The  practical  advantages  of  using  cloze  procedure  in  the  classroom 
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are  many  and  the  disadvantages  assuming  its  sensitivity  to  extended  assign—its 
seem  to  be  of  minor  stature.   First,  the  test  is  easily  constructed  and  saves 
the  teacher's  time  and  effort  in  assembling  it.    Also,  administration  of  the 
cloze  procedure  is  quite  simple,  and  it  fits  easily  into  the  time  schedule  of 
one  class  period.    Grading  is  also  time-saving  as  a  cloze  test  can  be  scored 
quickly  by  a  key.   The  mechanical  advantage  of  the  cloze  procedure  brings  out 
an  important  point.    The  teacher's  time  that  is  spent  preparing,  administering 
and  grading  a  test  is  reduced  considerably  by  using  cloze  procedure,  and  he 
has  much  more  time  to  prepare  for  classes,  counsel  students  and  actually 
teach. 

However,  two  disadvantages  seem  to  block  the  way  of  the  teacher  using 
cloze  procedure.    First,  students  do  not  seem  to  understand  the  merits  of 
cloze  procedure j  therefore,  they  must  be  convinced  of  its  effectiveness. 
This  report,  along  with  the  other  studies  mentioned,  hopes  to  add  more 
substantiation  to  the  use  of  cloze  procedure  and  thus  help  remove  the  block 
against  this  type  of  test.   The  other  disadvantage  is  that  at  the  fifteen 
per  cent  level  of  cloze  testing  the  cloze  procedure  is  not  adaptable  to  text- 
book material  because  a  test  would  have  to  be  given  for  approximately  each 
chapter.   In  this  report,  the  four  per  cent  level  will  be  tested.    If  cloze 
procedure  does  significantly  measure  comprehension  at  this  level,  then  a 
test  can  be  given  for  approximately  every  five  chapters.   Thus,  cloze  pro- 
cedure will  be  quite  acceptable  for  textbook  material. 

The  student  who  is  tested  by  cloze  procedure  profits  considerably  for 
several  reasons.   First,  of  course,  the  student  is  forced  to  understand  the 
material  covered.    Probably  the  best  outcome  of  using  cloze  procedure  for 
the  student  is  that  he  must  change  his  study  habits.    Ho  longer  can  he  study 
the  main  points  of  the  material  and  expect  to  pass  the  examination.  The 
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student  most  study  to  understand  the  material  because  he  has  no  Idea  where  the 
cloze  passage  is  going  to  fall.   Alse,  the  need  or  practice  of  extensive 
memorization  will  be  eliminated  by  the  use  of  cloze  procedure.    It  would  be 
impossible  for  a  student  to  memorize  every  chapter;  so  he  must  study  for 
understanding,  which  is  a  prime  goal  of  learning. 

In  summary,  regarding  the  justification  of  this  report,  it  seems  that 
the  replication  and  exploratory  sections  can  give  considerable  insight  into 
cloze  procedure  as  a  testing  toolj  and  because  of  the  practical  advantages 
of  cloze  procedure  this  insight  could  prove  beneficial  to  the  classroom 
teacher. 

PBOCEDUIE 

The  "t"  test  of  the  difference  between  means  was  used  to  test 
Hypotheses  I  and  III.   The  Pearson  product  moment  correlation  was  used  to 
test  Hypothesis  II.   The  five  per  cent  level  of  confidence  was  used  for  all 
tests. 

DESIGS 

To  test  hypothesis  I,  experimental   and  control  groups  were  utilized. 
This  was  done  to  measure  the  amount  of  learning  that  took  place  as  indexed 
by  the  difference  between  the  mean  of  the  cloze  test  scores  for  the  group 
not  exposed  to  the  short  expository  article  and  the  mean  of  the  cloze  test 
scores  for  the  group  that  had  been  assigned  to  read  the  short  expository 
article. 

The  testing  of  hypothesis  II  was  accomplished  by  a  post-only  design. 
This  was  done  to  determine  the  correlation  between  cloze  and  multiple  choice 
tests.   Since  control  data  were  available  for  the  cloze  procedure  it  was 
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decided  to  collect  control  data  for  the  multiple  choice  test*  This  permitted 
a  test  of  the  learning  that  took  place  from  reading  the  short  expository 
article  as  indexed  ty  the  multiple  choice  test. 

To  test  Hypotheses  III  a  pre -post  design  was  utilized.   This  was  done  to 
permit  the  investigation  of  the  effect  of  prior  experience  with  cloze  procedure 
as  well  as  to  measure  the  amount  of  learning  that  took  place  from  reading  the 
extended  textbook  assignment.   The  subjects  receiving  the  pre-test  were  com- 
pared with  the  scores  of  the  post-only  experimental  group  to  determine  the 
effect  of  prior  experience  with  the  cloze  procedure, 

SUBJECTS  AND  DATA  COLLECTION 

The  subjects  consisted  of  128  Oral  Communication  I  students  from  seven 
classes.  These  students  were  enrolled  in  the  1966  summer  session  at  Kansas 
State  University,  Manhattan,  Kansas, 

The  pre-test  of  the  extended  textbook  assignment  was  administered  dur- 
ing the  first  class  meeting  of  the  summer  session  to  two  classes  with  six- 
teen and  eighteen  students  respectively.   The  cloze  test  used  for  the  pre- 
test consisted  of  fifty  deleted  words  from  three  randomly-selected  passages 

within  the  five  chapters  of  the  course  textbook,  Creative  Speech,  by  Keith 
16 

St,  Onge.       Each  passage  was  on  a  separate  page  of  the  test  booklet  so  the 
student  could  easily  determine  when  one  passage  ended  and  another  began. 
Every  tenth  word  was  deleted  from  these  separate  passages  creating  fifty 
blanks  for  the  students  to  duplicate  the  missing  words.   The  five  chapters 
consisted  of  approximately  13,270  words  and  the  cloze  passage  of  500  words 


Keith  St.  Onge,  Creative  Speech,  (Belmont,  California:  Wadsworth 
Publishing  Company,  196ii), 
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amounted  to  four  per  cent  of  the  complete  assignment.    The  students  were  not 
informed  of  the  test  source  material  but  were  informed  that  the  material  hand- 
ed them  contained  several  deleted  words  which  they  were  to  replace  with  the 
words  they  thought  were  correct.   They  were  given  two  cays  to  read  the 
chapters  of  the  extended  assignment,  and  then  the  post-tests,  which  were 
duplicates  of  the  pre-tests,  were  administered  on  the  third  class  day.  The 
experimental  post-only  class  consisted  of  seventeen  students  and  followed  the 
same  pattern  as  the  other  classes,  except  for  the  p re-test* 

The  test  materials  over  the  short  expository  article  consisted  of  cloze 
and  multiple  choice  tests.   The  cloze  test  way  given  to  one  control  class 
containing  nineteen  students  while  the  multiple  choice  test  was  given  to 
one  control  class  of  twenty-one  students.   The  cloze  test  once  again  con- 
tained three  passages  from  the  message,  and  a  total  of  fifty  deleted  words. 
The  multiple  choice  test  contained  twenty-five  multiple  choice  questions. 

The  short  expository  article  was  "Three  Misconceptions  of  Communica- 
tion" by  Francis  Cartier,  and  contained  approximately  2,800  words.  The 
500-word  cloze  passage  amounted  to  17 »8  per  cent  of  the  article.^ 

The  tests  were  administered  to  the  experimental  groups  one  day  after 
the  article  was  distributed  to  the  classes,  and  the  multiple  choice  tests 
and  the  cloze  tests  were  duplicates  of  the  test  material  administered  to  the 
control  groups.   Two  different  classes  containing  seventeen  and  twenty 
students  received  these  test  materials.    One  class  was  given  the  cloze  test 
first  while  the  other  class  was  given  the  multiple  choice  test  first  to 
cancel  any  order  difference  that  might  occur. 


Francis  A.  Cartier,  "Three  Misconceptions  of  Communication,"  A  Hs- 
vlew  of  General  Semantics,  XX,  (July  1963),  pp.  135-11*5. 
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IESULTS 

Generally,  this  section  is  arranged  according  to  the  hypotheses  under 
study.   However,  there  was  a  possibility  that  prior  experience  with  the  cloze 
procedure  would  inflate  cloze  scores  obtained  after  exposure  to  the  experi- 
mental message  (for  either  the  short  expository  article  or  the  extended  text- 
book assignment).  While  the  design  utilized  experimental  and  control  groups 
for  the  testing  of  the  hypotheses,  it  permitted  an  investigation  of  this 
variable.   The  mean  of  the  post-test  scores  for  the  control  group  for  the 
extended  textbook  assignment  was  compared  with  the  mean  of  the  test  scores 
for  the  experimental  group  for  the  extended  textbook  assignment  (see  Table  I), 

TABLE  I 

Pest  Control  and  Post  Experimental  Mean  Differences 
for  Extended  Textbook  Assignment 


aroup  tmm  H  t 

Post  Control  22.61 

Post  Experi-  19.88 
mental 

*t  .9$  (df  s  h9)  a  2.02 


While  the  analysis  did  not  reveal  a  significant  prior  exposure  effect 
the  obtained  "t"  score  approached  significance  and  all  analysis  related  to 
the  hypotheses  of  this  study  utilize  the  post-only  experimental  groups  and 
control  groups. 

Hypothesis  I  predicted  that  there  would  be  a  difference  between  the  mean 
of  the  cloze  scores  of  the  students  who  read  the  short  expository  article  and 
those  who  did  not.   This  hypothesis  is  supported  by  the  data  (see  Table  II). 
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TABLE  IT 

Control  and  Experimental  Mean  Differences  for  Cloze  Test 
Over  Short  Expository  Article 


Group  Mean  N  t 

Control  12.52  19 

li.61* 

Experimental  18.51  37 

*t  .95  (df  =  5U)  «  2.00 


Hypothesis  II  predicted  significant  correlation  between  the  scores  of 
the  multiple  choice  test  and  the  cloze  test  over  the  short  expository  arti- 
cle.  The  obtained  r  of  .20  was  not  significant  (r  .95  -  .32,  df  s  56). 

Since  no  significant  correlation  was  found  but  support  was  obtained  for 
Hypothesis  I  which  predicted  a  significant  increase  in  cloze  scores  as  a 
result  of  exposure  to  the  short  expository  article,  a  similiar  analysis  was 
conducted  for  the  multiple  choice  test.   Again,  there  was  a  significant 
increase  in  multiple  choice  scores  as  a  result  of  exposure  to  the  short 
expository  article  (see  Table  III). 

TABLE  III 

Control  and  Experimental  Mean  Differences 
for  Multiple  Choice  Tests  Over 
Short  Expository  Article 

Group  lean  R  t 

Control  10.66 
Experimental  15.6k 
*t  .95  (df  =  56)  s  2.00 
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37 


5.531 


Bypothesis  III  predicted  that  the  cloze  scores  of  the  students  who  had 


read  the  extended  textbook  assignment  would  be  different  than  those  who  had 
not  read  this  assignment.   This  hypothesis  is  supported  by  the  data  (see 
Table  17). 

TABLE  17 

Close  Pre  and  Post  Only  Scores  of  the 
Extended  Textbook  Assignment 


Group  Mean  N  t 

Pre  Cloze  15.97 

Post  Only  19.88 
Cloze 

*t  .95  (df  =  1*9)  =  2.00 


DISCISSION 

The  support  given  to  Hypotheses  I  and  III,  and  the  non-support  of 
Hypothesis  II  brings  forth  several  points  that  need  further  explanation. 
First,  by  supporting  Hypothesis  I,  substantiation  was  added  to  Taylor^ 
findings  that  cloze  procedure  does  measure  reading  comprehension  over  a 
short  expository  article.   Also,  by  supporting  Hypothesis  IU  the  apparent 
limitation  of  a  fifteen  per  cent  cloze  passage  does  not  seem  to  be  a  limita- 
tion at  this  point  in  the  substantiation  of  cloze  procedure. 

However,  the  non-support  of  Hypothesis  II  does  not  add  substantiation 
to  Taylor" s  findings  that  cloze  procedure  tests  will  correlate  highly  with 
other  objective  tests.   The  non-support  of  this  hypothesis  shows  that  cloze 
procedure  will  not  correlate  with  saw  other  objective  tests,  and  does  raise 
the  point  that  cloze  procedure  cannot  be  substituted  for  other  objective 
tests  in  some  circumstances. 


3k 
1? 


2.92* 


21 


HESEABCH  IMPLICATIONS 

This  report  shows  that  in  several  places  of  the  substantiation  of  cloze 
procedure  that  more  research  is  needed*   First,  the  support  of  Hypothesis  in 
points  out  that  research  is  needed  between  the  levels  of  four  per  cent  and 
twenty  per  cent  of  the  complete  article  being  read,  to  substantiate  that  cloze 
tests  do  measure  between  these  two  levels.    Further  exploratory  hypotheses  do 
net  seem  justified.   Some  work  in  the  development  of  a  rationale  for  the 
applicability  of  cloze    procedure  to  manuscripts  of  varying  lengths  when  cloze 
procedure  is  used  as  a  measure  of  comprehension  is  needed*   The  non-support 
of  Hypothesis  II  implies  that  more  research  is  needed  to  find  what  types  of 
objective  tests  cloze  procedure  scores  will  correlate  highly  with,  and  in 
effect  if  cloze  procedure  is  measuring  the  same  thing  as  other  objective 
tests*   The  space  between  the  deleted  words  may  be  a  critical  factor  in  the 
correlation  problem*   Thus  it  may  be  that  by  deleting  every  tenth  word  the 
correlation  is  affected.   This  seems  to  be  the  starting  point  on  further 
research  on  the  correlation  problem. 

One  other  point  that  pertains  to  further  research  is  that  scholars  who 
use  cloze  procedure  as  a  measuring  device  in  their  studies  should  report  the 
level  of  cloze  procedure  that  was  used,  and  the  space  between  deletions.  This 
would  increase  the  utilization  of  research  on  cloze  procedure  as  these  fac- 
tors could  be  analyzed  at  the  same  time  the  study  in  general  is  being  read. 

In  summary,  this  report  gave  support  to  Taylor's  earlier  findings  that 
cloze  procedure  measures  reading  comprehension  over  a  short  article,  and  it 
lends  support  to  the  position  that  cloze  procedure  does  measure  reading  com- 


prehension  over  a  longer  reading  passage.  But,  no  support  was  obtained  for  the 
position  that  cloze  procedure  correlates  with  objective  tests* 

One  limitation  of  cloze  procedure  Is  that  students  react  negatively  to 
new  testing  procedures.    It  is  hoped  that  this  report  may  help  with  the  chang- 
ing of  these  negative  attitudes. 
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AX  EtPEKDEWAL  STOUT  OF  CLOZE  PiXXJEDOHE 
AS  AH  IHEEX  OP  HEADING  OOMPflEHEHSlOH 

The  objective  of  this  report  was  to  add  or  subtract  support  to  Wilson 
Taylor*  s  findings  that  cloze  procedure  Is  an  effective  measurement  of  read* 
lag  ©emprehension,  and  to  explore  the  application  of  close  procedure  to  text- 
book reading  assignments* 

The  review  of  the  literature  revealed  that  cloze  procedure  is  an 
accepted  method  of  measuring  reading  comprehension*   However,  cloze  procedure 
research  had  been  limited  to  short  articles* 

The  hypotheses  of  this  study  worst 

I.    Close  procedure  scores  measure  reading  comprehension 

of  a  given  short  expository  article. 
II*   Closs  procedure  scores  correlate  significantly  with 

a  multiple  choice  test  over  a  given  short  expository 

article* 

III*  Close  procedure  scores  measure  reeding  comprehension 
over  a  given  extended  textbook  assignment* 

Data  were  collected  by  administering  close  sad  multiple  choice  tests 
to  Oral  Communication  I  students  at  Kansas  State  University  in  the  summer 
session  of  1966 »  The  design  consisted  of  an  experimental  and  control  group 
being  given  a  cloze  test  before  end  after  they  read  the  experimental 
messages*  Cloze  procedure  scores  and  the  multiple  choice  test  scores  were 
correlated  and  the  significance  of  the  difference  between  control  and 
experimental  groups*  means  were  determined* 

The  obtained  data  were  analysed  by  the  use  of  the  "t"  test  and  the 


Pearson  product  moment  correlation* 

Hypotheses  I  and  III  were  supported  by  the  findings,  and  Hypothesis  II 
was  rejected. 


